Skip to content

feat(desktop): add Agent Usage UI backed by the NIP-AM local archive#2035

Draft
wpfleger96 wants to merge 9 commits into
mainfrom
duncan/agent-usage-archive
Draft

feat(desktop): add Agent Usage UI backed by the NIP-AM local archive#2035
wpfleger96 wants to merge 9 commits into
mainfrom
duncan/agent-usage-archive

Conversation

@wpfleger96

@wpfleger96 wpfleger96 commented Jul 17, 2026

Copy link
Copy Markdown
Member

Adds a rolling token/cost usage view for agents, sourced from the local NIP-AM metrics archive. Covers both the Agents overview and a focused per-agent subview in the profile panel, with the per-agent breakdown grouped by (harness, model) so the same model running under two different harnesses (e.g. claude-sonnet-4-5 via goose vs claude-code) renders as two distinct rows.

Backend

  • get_agent_usage_series Tauri command (archive/agent_usage.rs): reads the local SQLite archive for a rolling 7d/30d window, returns per-agent token and cost totals broken down by (harness, model), with partial/unknown-field flags when evidence is incomplete. Sort tiebreak: harness ascending → model ascending, None last in each.
  • agent_metric_index: nullable harness TEXT column, parsed from AgentTurnMetricPayload.harness and written/read through all store paths. Schema migration M1 runs as a single SQLite transaction — schema-guarded ALTER TABLE, full index rebuild from the canonical archived_events store (ingest and backfill share the same from_payload parser), completion marker written last — so a crash mid-migration can never leave a half-built index marked complete.
  • Wired persistedAgentMetrics notifier through the archive sync path so new metric events are picked up without a full resync.

Frontend

  • AgentUsageSection (features/agent-usage/ui/): ranked agent list in AgentsView, between the unified agents list and teams section. Row click-through opens the profile panel focused on usage.
  • AgentUsageFocusedView: per-agent 7d/30d totals with a by-(harness, model) breakdown, rendered as a focused profile-panel subview (same pattern as Memories/Diagnostics) rather than a new tab. Each breakdown row shows the harness as a dimmed sub-label next to the model name; null harness (pre-migration or unknown data) renders no label, so single-harness data is visually unchanged.
  • UserProfilePanel/UserProfilePanelSections/UserProfilePanelTabs: threaded canViewUsage/onOpenUsage (owner-only, bot profiles only) and added a BarChart3 ingress row in the Info tab.
  • agentUsage.ts sortModelsByKnownTotal: ordinal compound (harness, model) tiebreak matching the Rust ordering (identifier domain is ASCII; documented).
  • Loading skeleton, query-error retry, empty state, and a collection-off banner (with retained-data coverage copy when historical data still exists) are all covered.

Tests

  • Rust: unit tests for bucket-boundary math, cost-ladder aggregation (direct/cumulative/decrease-taint), ranking tie-breaks (including harness), the same-model/two-harness collapse fix, and out-of-range request validation; integration tests exercising the command against a real SQLite archive (fresh ingest, pubkey filtering, pre-existing-row backfill); store_migration_tests.rs covering M1 through the real open_archive_db path (legacy-file backfill, idempotent reopen, crash-before-commit recovery).
  • agentUsage.test.mjs: pure-helper unit tests (formatting, partial/unknown-field detection, compound-key sorting).
  • tests/e2e/agent-usage.spec.ts: loading → resolved, window switch, both click-through paths into the focused view, error/retry, empty state, collection-off banner (with and without retained-data copy), partial-total badge, and harness sub-label rendering.

@wpfleger96
wpfleger96 requested a review from a team as a code owner July 17, 2026 17:08
@wpfleger96
wpfleger96 marked this pull request as draft July 17, 2026 17:10
@wpfleger96
wpfleger96 force-pushed the duncan/agent-usage-archive branch from 3bdb2a1 to 2451115 Compare July 20, 2026 17:45
npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 and others added 3 commits July 20, 2026 23:20
Adds the Rust backend half of the NIP-AM local agent usage feature
(Rev 3 frozen plan): a rebuildable agent_metric_index parsed from
archived kind-44200 rows, a pure per-field accounting ladder
(agent_usage.rs) computing token/cost deltas with adjacent-cumulative
preference and direct-value fallback, and the get_agent_usage_series
Tauri command wiring backfill, orphan repair, collection-enabled
detection, and A13's hasArchivedEvidence into one series response.

commit_archive now indexes kind-44200 rows in the same transaction as
the canonical event insert, and upsert_archived_event/gc_orphaned_events
return/enforce the new-row and cascade-delete invariants (A5/A6) that
persistedAgentMetrics and the index's rebuildability depend on.

get_agent_usage_series' SQLite core is split into a plain sync fn so
it can be driven directly against an in-memory Connection in tests
without a Tauri AppState.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Completes Phase 2 of the NIP-AM local agent usage feature. tauriArchive
decodes the backend's persistedAgentMetrics count defensively (missing
field -> 0, per A5/M2's backward-compatible wire contract) and exposes
an onAgentMetricsChanged notifier, fired only when a subscription
mutation touches kind 44200 specifically.

ArchiveSyncManager routes both its idle/size-triggered and destroy-time
flushes through one sendBatch helper that notifies usage listeners
only when the backend reports persistedAgentMetrics > 0 — rejections,
duplicate-only batches, and non-metric successes never notify, keeping
the refresh signal backend-authoritative.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Surfaces per-agent NIP-AM turn-metric usage (tokens, cost, model
breakdown, 7d/30d bucket series) in the desktop app, reading from the
Phase 2 get_agent_usage_series archive command:

- agent-usage/lib/agentUsage.ts: pure client-side helpers -- bucket
  boundary construction, bigint-safe token formatting, and the ranked
  overview-row projection shared by both UI surfaces.
- agent-usage/hooks.ts: react-query wrapper around
  get_agent_usage_series with the app's standard retry:1 default.
- agent-usage/ui/AgentUsageSection.tsx: ranked per-agent usage rows on
  the Agents overview, with a 7d/30d window switch, retry-on-error,
  and a collection-off banner (with or without retained-data coverage
  copy) linking to Local Archive settings.
- agent-usage/ui/AgentUsageFocusedView.tsx: per-agent drill-down
  (bucketed series + model breakdown) reached via an overview row or
  the profile panel's Info-tab usage ingress row; wired into
  AgentsView, UserProfilePanel, and ProfilePanelContext.
- Partial badge on rows whose total is a known lower bound
  (has_unknown_usage), surfaced via the accounting engine's per-field
  completeness contract.

Extends the Rust archive test suite (agent_usage_tests.rs) to close
gaps in the pure accounting engine: the f64 cost ladder had no direct
outcome.cost assertions, assign_bucket_index's start-inclusive/
end-exclusive edges were only exercised indirectly, validate_request's
chrono finite-range rejection was unexercised, and the A2 ranking
tiebreak (pubkey ascending on an exact totalTokens tie) had no test
distinguishing it from insertion order.

tests/e2e/agent-usage.spec.ts (wired into playwright.config.ts's smoke
project) exercises both UI surfaces against a mocked
get_agent_usage_series: ranked rows, window switching, click-through
to the focused view from both the overview row and the profile
panel's ingress row, retry recovery from a query error, the
collection-off banner and its settings deep link, and the Partial
badge.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wpfleger96
wpfleger96 force-pushed the duncan/agent-usage-archive branch 2 times, most recently from 4576ca6 to b213a00 Compare July 21, 2026 07:00
…chive

* origin/main: (25 commits)
  feat(cli): filter archived instances from --template roster resolution (#2207)
  chore(release): release Buzz Desktop version 0.4.21 (#2209)
  fix(desktop): clarify data deletion action (#2208)
  feat: brand authentication complete page (#2192)
  fix(buzz-acp,buzz-agent): surface stall duration and fate (#2204)
  fix(desktop): resolve activity feed showing channel UUID instead of message content (#2201)
  feat(cli): add agents archive/unarchive/archived subcommands (#2173)
  chore(acp): strip stale finding-number references from comments (#2202)
  chore(release): release Buzz Mobile version 0.4.9 (#2200)
  Simplify first-community onboarding choices (UI only) (#2194)
  fix(mobile): sanitize Android image uploads (#2188)
  fix(cli): paginate channel directory queries (#2181)
  Fix persisted agent defaults display (#2182)
  [codex] Fix missing release tag detection (#2191)
  feat(web): authenticate gated community browsing (#2190)
  fix(timeout): unified turn-timeout fix — cap inheritance, steer renewal, activity-aware requeue, LLM stall surfacing (#2175)
  fix(desktop): scope draft store per workspace relay (#2179)
  chore(release): release Buzz Mobile version 0.4.8 (#2187)
  [codex] create release tags with dedicated App (#2186)
  fix(desktop): avoid forwarding click event as channel callback (#2174)
  ...

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wpfleger96
wpfleger96 force-pushed the duncan/agent-usage-archive branch from b213a00 to 122b043 Compare July 21, 2026 08:04
wpfleger96 pushed a commit that referenced this pull request Jul 21, 2026
wpfleger96 pushed a commit that referenced this pull request Jul 23, 2026
Four Playwright tests covering the new Agent Usage UI surfaces:
01 overview card (default/collapsed), 02 focused usage subview,
03 multi-day bars chart, 04 empty state (collection on, no data).

Also adds agent-usage-screenshots.spec.ts to the smoke project
testMatch so CI can discover and run it alongside the other
screenshot specs.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Stack: #2035 → this PR

## What

Group the per-agent model breakdown by `(harness, model)` instead of
`model` alone, so the same model running under two different harnesses
(e.g. `claude-sonnet` via `goose` vs `claude-code`) produces two
distinct rows rather than collapsing into one.

## Changes

**Rust / SQLite**
- `agent_metric_index`: add nullable `harness TEXT` column. Schema
migration M1: `ALTER TABLE ... ADD COLUMN` + delete-then-backfill
rebuild so all existing rows get `harness` populated from retained
`archived_events.raw_json` — no data loss; ingest and backfill share the
same `from_payload` parser (frozen-plan requirement preserved).
- `AgentMetricIndexRow`: parse `harness` from
`AgentTurnMetricPayload.harness` (REQUIRED field per NIP-AM), write/read
through all store paths (`ROW_COLUMNS`, INSERT, `row_from_sql`).
- `AgentScope.models` grouping key widened from `Option<String>` to
`(Option<String>, Option<String>)` i.e. `(harness, model)`. Sort
tiebreak: harness ascending → model ascending, `None` last in each.
- `ModelUsage` wire type: add `harness: Option<String>` field.

**Frontend**
- `tauriArchive.ts` / `bridge.ts`: add `harness` to `AgentUsageModel`
type.
- `AgentUsageFocusedView`: render `harness` as a dimmed sub-label next
to each model name on breakdown rows. `null` harness (pre-migration data
or unknown) renders no label — single-harness data is visually
unchanged.
- `agentUsage.ts` `sortModelsByKnownTotal`: tiebreak updated for
compound `(harness, model)` key.

**Tests**
- Rust: collapse-fix test (same model / two harnesses → two rows),
migration test (old-shape rows get harness populated after rebuild),
harness sort coverage.
- TS: `sortModelsByKnownTotal` harness tiebreak tests, same-model
two-harness sort test.
- E2E: `bridge.ts` fixture type updated, `agent-usage.spec.ts` harness
assertion, `agent-usage-screenshots.spec.ts` shot 02 harness-label
visibility check.

## Gates

All green locally:
- `just desktop-check` ✓
- `just desktop-typecheck` ✓
- `just desktop-build` ✓
- `just desktop-test` (3487 pass, 0 fail) ✓
- `cargo test` in `desktop/src-tauri` (1575 pass, 0 fail) ✓

---------

Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
@wpfleger96

Copy link
Copy Markdown
Member Author

Overview usage card

Daily bar chart and agent row showing token totals with a PARTIAL badge for the current period.
01-overview-usage-section

Focused usage view — (harness, model) breakdown

Same model under two different harnesses renders as two separate rows with dimmed harness sub-labels: claude-opus-4-5 claude-code and claude-sonnet-4-5 goose.
02-focused-usage-view

Multi-day bars

Multi-day bar chart spanning several dates with per-day token counts and a full agent row.
03-daily-bars-multi-day

Empty state

Empty state message shown when no locally archived usage exists in the selected period.
04-empty-state

wpfleger96 pushed a commit that referenced this pull request Jul 25, 2026
npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 and others added 3 commits July 25, 2026 15:43
…te formatter

The branch had ratcheted the desktop file-size gate four times. One of those
entries re-declared `src-tauri/src/lib.rs` at 1001 while a pre-existing entry
at line 58 sets 1013 — the override map is a JS `Map`, so the later key won and
the branch silently lowered a ceiling it never needed to touch (lib.rs is 920).

Splitting at existing seams removes the other two: the M1 schema migration
moves to `archive/store_migrations.rs` behind one `pub(super)` entry point, and
the kind-44200 archive tests move to `archive/mod_agent_metric_tests.rs`,
`#[path]`-included from `mod_tests.rs` so they keep reaching the shared
fixtures through `use super::*`. `mod_tests.rs` returns to its pre-branch 1208
ceiling rather than dropping it, since that debt predates this work.

`formatCoverageDate` was duplicated verbatim in both usage components; it now
lives beside the other formatters in `lib/agentUsage.ts`.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…chive

* origin/main: (112 commits)
  docs(contributing): trim to goose-scale minimal intake surface (#2780)
  fix(relay): preserve reconnect backoff (#2759)
  refactor(relay): expose reconnect timing policy (#2310)
  fix(desktop): clear stale working badges on agent stop/restart (#2803)
  fix(desktop): surface agent rename relay profile sync failure as a warning toast (#2279)
  fix(docker): create /data/git so the compose volume inherits buzz ownership (#2840)
  fix(mobile): invalidate DM directory providers at the community boundary (#2842)
  feat(relay): make per-owner community limit configurable via BUZZ_MAX_COMMUNITIES_PER_OWNER (#2599)
  fix(discovery): inject PATH into Codex adapter planning (#2767)
  chore(release): release Buzz Desktop version 0.4.26 (#2808)
  Refine mobile navigation and creation flows (#2810)
  feat(relay): add author-only-unless-shared read gate for kind 30175 (#2768)
  fix(core): block IPv6 transition SSRF targets (#2801)
  Style mobile pairing QR codes (#2775)
  Refine community management flows (#2738)
  fix(workflow): bypass system proxies for webhooks (#2800)
  docs: replace VPN-vendor references with generic wording (#2805)
  docs: point readme at deploy compose bundle (#2363)
  fix(desktop): explain macOS local network access (#2263)
  fix(desktop): clarify CLI runtime setup (#2680)
  ...

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Merging main brought in #2098, which removed the additional-agents gallery and
shrank `UserProfilePanel.tsx` from 1014 to 983 lines. This branch's additions
land it at 1009 — under main's existing 1025 ceiling — so the ratchet to 1040
is no longer load-bearing.

The branch now leaves the override map byte-identical to main: no new entries,
no raised ceilings, no lowered ones.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant